Augmented Reality (AR) is a technology that enhances a person's view of the real world with virtual objects such as imagery and textual data. While a virtual reality system places the user in a totally synthetic computer generated environment, an AR system merges computer synthesized objects with the user's space in the real world. In an AR system computer generated graphics enhance the user's interaction with and perception of the real world. For example, in industrial operations different entries in a factory database may need to be accessed depending on the user's location within the factory environment. Real time operational data and maintenance information may be stored on a remote server that is accessible at the factory site. Ideally a user could access and view manuals and current operating parameters of various components and equipment as the user moved through the factory. Similarly, this type of information could be viewed remotely by engineers and technicians when maintenance is being performed by an automated device or in a training scenario. In a particularly large factory the user may also need to be guided to an area of interest. An example of the use of augmented reality in such environments is disclosed in CYLICON: A SOFTWARE PLATFORM FOR THE CREATION AND UPDATE OF VIRTUAL FACTORIES by Navab et al., Proceedings of the 7th IEEE International Conference on Emerging Technologies and Factory Automation, pp. 459-463, Barcelona, Spain 1999.
A typical augmented reality system includes a display device and a tracking device with associated software housed in a mobile or wearable computer such as the Sony Vaio Picture-Book and the Xybernaut MAIV wearable computer. Such computers can be either worn or hand held and connected to a computer network via wireless communication. The software monitors tracking device events in order to determine the present position of the display in the real world and to retrieve virtual objects for use by the viewer. In order for the display to present the correct virtual objects, the virtual objects and the real world need to be in registration or synchronized in some fashion. A virtual object should appear at its proper place in the real world so that the user can correctly determine spatial relationships. Registration of the computer generated graphics should be dynamically adjusted in response to changes in the user's real world perspective.
Registration implies that the geometry of the virtual camera which is retrieving the augmentation data is known with respect to the real world. To be effective the tracking device must provide extremely accurate data concerning the real world view in order to ensure seamless rendering of the virtual objects as they are superimposed over the real world view. In typical state of the art AR systems, a virtual object often appears to waver or drift as the user moves, and does not appear to rest at the same location as the user views that location from several different positions. These defects in registration are typically due to shortcomings of the tracking system.
Many tracking systems use magnetic trackers, such as disclosed in U.S. Pat. No. 6,262,711, entitled COMPUTERIZED INTERACTOR SYSTEMS AND METHOD FOR PROVIDING SAME, issued to Cohen et al. Conventional magnetic trackers may be subject to large amounts of error and jitter and can exhibit errors on the order of ten centimeters, particularly in the presence of magnetic field disturbances such as metal and electrical equipment commonly found in factories. Carefully calibrating a magnetic system typically does not reduce position errors to less than two centimeters.
Other AR tracking systems use image recognition to track movement, and nearly perfect registration can be achieved under certain conditions. An example of an image or vision based system is disclosed in U.S. Pat. No. 6,330,356, entitled DYNAMIC VISUAL REGISTRATION OF A 3-D OBJECT WITH A GRAPHICAL MODEL, issued to Sundareswaran et al. Under some conditions such image recognition systems can become unstable. Instability usually originates with software embedded assumptions, which may or may not be accurate, that are made about the working environment and the user's movement in order to reduce computation costs.
Numerous attempts have been made to solve the registration problem. U.S. Pat. No. 6,064,749, entitled HYBRID TRACKING FOR AUGMENTED REALITY USING BOTH CAMERA MOTION DETECTION AND LANDMARK TRACKING, issued to Hirota et al., discloses the use of a concentric landmark or marker for use in conjunction with image recognition. The landmark includes a first dot of a first color and a ring concentric to the first dot. The ring is of a second color which is different from the first color. Typically the diameter of the ring is about three times the diameter of the dot. The Hirota et al. device includes an image analyzer that first views an image in search of areas whose color matches the outer ring of a concentric landmark and the attempts to locate the inner colored dot within the identified area. The applicants of the present invention have found through experimentation that the color coding of markers often results in instability of the classification protocol due to frequently changing illumination, such as might occur when moving from place to place within a factory environment.
Hoff et al. at the Colorado School of Mines has also developed an observer pose determination system based on concentric circular markers. See Hoff, W. A.; Lyon, T. and Nguyen, K. “Computer Vision-Based Registration Techniques for Augmented Reality,” Proc. of Intelligent Robots and Computer Vision XV, vol. 2904, in Intelligent Systems and Advanced Manufacturing, SPIE, Boston, Mass., pp. 538-548 (1996). By processing a video image of the object with the markers in place the markers are isolated. Hoff et al. then uses an estimation algorithm to estimate the pose of the camera. These particular markers are cumbersome and require excessive computational resources.